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' ABSTRACT 

The %ajor issues encountered by the testing staff of 
the Austin Independent School' District when changing achievement 
testis were identified. Th§ 1970 California Achievement Tests were 
Replaced by the district with the 1978 Iowa Tests of Basic Skills. 
The major tasks were selection of a new instrument and securing funds 
to purchase it. However, during the transition , many smaller # . 
decision^ must be made. The first is. a determination that the old 
test is .useless. A new test must be selected and money for its # 
purchase secured. All staff concerned should be notified of the 
change. The curriculum ^should, be reviewed before the new test is 
selected.. Criteria and^rercfquisites must be updated. Changes in 
norming dates £nd test* content nee4 to be considered as well as] 
"revised forms and procedures. ^fcw reports should be designed*, for test 
scores. Staff -must be „ trained' in the use of , the new instrument. ' 
Longitudinal comparison should "be resolved. The processing and 
5 scoring system should be redesigned. Old materials should.be * 
. completely discarded and ^security measures should be prepared so that 
the access io tests is limited. • Effective planping and*a~ successful 
transition can octfur*if these lesser decisions are anticipated. 
(DWH) , ' • \. • , , " 
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. When your test booklets $re all dog-feared and .coming unstapled, when 'you have 
marked, all five answers- to some items in the booklets because students- have 
marked tfre "correct" fctioice, when your college-bqund seniors are scoring at the 
25th percentile because you ate still using 1965. norms, when half the teachers 
have copies of the test in*their desks, and when* older students support their 
' habits by selling test items to freshmen, then you finally get* up the nerve to 

face changing achievement tests; Every ^ounce of energy* is focused on two tasks — 
f selecting A replacement and' obtaining the money to purchase itX When this \ 
' happened in the^ Austin Independent School District,- we discovered that these 
two tasks were"3^st the tip of ah iceberg. - Literally hundreds of smaller tasks, , 
issues, *and decisicjns* loomed below* * j 

The selection pf a new achievement test and the securing of the funds to purchase 
"it are indeed the overriding concerns of systems ^hanging tests . During the 
transition, hundreds of smaller decisions must be made — many of these involving 
changes necessitated by the' new Jtest but Ijunant^ipated beforehand. To ensure 
a duccesgful* transition, these decisions musf^ue anticipated to allow planning 
to take place. The expediences of 'Austin's School system can be of great 
assistance to others, not so much in providing answers, as in identifying the 
issues which* must be addressed. . * • " • 

I Not impressed by the iceberg analogy?/ Try tfcis one. Did anyone ever, give you 
' 'a tie or a scarf, and. you had^to buy a shipt/feo-match it? Then you had to buy 

# - pants or a skirt to match 'that. Next, you had to buy a be^t, then shoes — how 
/about a'hat? Welcome to the chain- reactibn world of accommodating a new achieve- 
ment test. - , 4 

* * • * 

The attached "floe" chart "graphically sumtoarizes the majQr issues which were 
encountered 3 by the # testing staff' in Austin when the 1970 California Achievement 
Tests was replaced, By the 1978 Iowa Tests of Basic Skills , Some further de- 
ta'ils of each i^sue are presented below. 1 *| 

4 * • • 

•• '* ' •• s- " 

"details of the' hajor tasks in changing ACHIEVEMENT TESTS, v 
] . I. Determine that old test is useless. 

* v \ *' II. Select a new test. ' 
: $L •*« * *S6cure money for purchasing the. new tefet. \ " ' 

£ , V Itf* Inform everyone. There are more people to inform than anyone' * realizes, 

* °. . 'V .and they each, expect to be^the;.|irst to know since they have their own * ., 
*t* * t% icebergs* Not informing Everyone early-on wijl haunt you all year. 

\ v# 0 v M A* .Decision process:' Why was- a new tdst neecfed? Why in \ ' 9 

\ v , \\ V th ? w <5rld did you choose THAT one?(! 'Who had input into**' 

' ^ / S **** decision? Who made, the final decision? „ 
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t f — ' — " - - • 
B* Areas to be tested:' Are they the k same? What arfe they? 

. C. Testing dat$s: When' will the tests be given? Is tjhis 
a change? 

p.. Release of results: Wen will the results. be released? 
Why not* earlier? ^ 

A.-D. . 

1. parents (public) . »• * 



a. Newsletters 

b. News releases * * 0 

c. Public service announcements (radio, TV) 

* 2. Staff - . 

% a. -Staff publications 

b. Announcements 

c. .Official infernos 

d* Official* calendars- - • , 

V. Review cur riculum. ^Teachers pick up on this -early — the new tefet 
does not measure anything from their favbritte.unifc} or the new test • 
measures map reading skills, and their students cannot yet locate the 
exit during fire drills. # x V 

* A. Relative emph e asis on jslcills: Ho£ many items measure 
each skill, pfci the new test? 

B; Terminology^ What Vocabulary and terms are different? ^ , 
Me f oimd, that^^%orrowin k g;! , ^l our math books was called" 
"renaming" on the tes t and chat blanks -were substituted • 
fo.r 'commas in long numbers. The students were probably . a 
* nonplussed by this; the instructional staff took this # , 
very- seriously. * 9 ! \'\ 

C# Skills not currently taught: Are new skills now'^ing 
^ # to be tested? Baise 4 r Ine trie measures, map reading — we 0 • 
found several others, tooV • \ • ' 

IV. Update criteria and prerequisites. * Changes in norming 1 4<ates and test % 
content combine to affect the number of students whb> ar^ selected* by 
existing criteria. A conscious decision to accept a shift in thd ' * 
impact of established criteria or to change .criteria to keej^a constant 
number of students in/out of programs is necessary* . f. 

A# Course prerequisites: ^Should the new test be use^- to^- \ ^ • 
# . replace existing/placement tests?; We. used the '.change* \ ;« 

" in tests; as BfX " opMMjt^nJtty to droprsepafca£e language arts ^ 

and sciejate; plac^snfr" tMts.lnfavor^of siAsdalesVon th6 - . r 
^ new test* ; We also toaji peirceAt^le jp'r%r$qxi±sites fbr somfi ^ . „ 
reading and math courses. \We* rie.tained'the established '-s ; 
' criteria—probably not. the^fetter choice* in our Situation, \ '* , 
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B. Minimum competency requirements for. graduation; Should 
.-the requirements be changed to match the new areas tested 

or the tiew norms?" Oiir change' in- tests coincided with a 
' , raise* by ttie schoo v l board in the grade level equivalent - 
* fot mihimum competency. ..The impact of e'ach change is 
unknown^ confound it! 

C. Goals arid objectives:; Do , stated goals and objectives \ 
* * need N revision to match new test areas and norms?! 

• / . 



1. ' Campus-level objectives 

2. Program objectives 
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3. State accreditation goals • • 1 \ 

' • ■■ A • 

D. Compensatory programs; D§ selection and 
revisions? % • / 



i 

d exit criteria need 



1. Title I (alias Chapter I) programs „ \ 

2« limited English proficiency programs , * 

3. State and locally funded programs 

Revised forms and^procedures. Do not underestimate tfiris one. Everyplafce- 
*we jLooked there was a test 'name or some related information that had to 
be 'revised. V « 

' I V r ■ 1 " ' K 

lL fAdministratiSn dates v and* times: When will the* tfest be 

administered? > • ' • 

1* Critical norming dates — iprobably different from * • 
the last test's * * & . 

, 4 .. r . r V • ♦ ' 

2. Length of individual tests ' 

§> • * * 

— a* JDays of the week 

b. v Mornings or 'afternoons 

B. Test l^els; Which ielvel is best for each grade? Will • 
out-of-leveX testing be allowed? What criteria- will be, 
usedifotf determining the need f or 'out-of-level testing? 

c f fG. Answer ctbcuments: Will the publisher's answer sh'eeta be 

* * used or will unique ones be -designed? When must they be 
A a -order of printed? / • " * 

D« J Practice .test: Will- the publisher's boused or will a . 
/ * locally developed one be needed? * Which grades and cate- «• 
• */$OTies v of' students shpoLd take a jjcactlce test? We ^ ' / 
developed^ our own*to use the first yea*r since the tesr 
fdrmat and out-of-level directions were new to outf : . : ¥ 

• . . students* ' Now we cannot' get the schools to give it* up; ' * 
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* * * y 

VIII* Design new l reports. * Changing tests not only necessitates new reports, \ 
tut also provides the opportunity -for-m^jor changes 'if desired. 

I . * ' ' ' ' 4 > v 

• A, Teachers . (student reports, classroom summaries) 



B. Principals 



(school 



summaries) 



C. Permanent records (student reports) f - \ 

D. Parents, public (student reports, school and district summaries) 
* E., Administration, School Board* (school and district summaries) 



A*-E. 

. * . 1., Brochures ~ * 

2. Printouts , 

■* * * 

3. Labels s • ■ 

» ■£ 

V „ 

9 ** » * 

i 4. Summaries 

' 1.-4. 

a. Norms *. 

» ' 0 b. ♦ Skilly are^s 

. ' c. Students excluded, included 

IX. Tr^in staff. .This is a 'key .task,. Many of the teachers 1 , counselors 1 , 
and principals 1 established routines are being changed, and they will 
npt; l^arn every h^w procedure tftfcough written communications. 




A» New procedures 



I 



B. . Curriculum 



4 



revisions 



ERIC 



C. New crit.erfa and' prerequisites 

' ° . ' r 9 \ 9 ' 

D. New tetmihology 

£♦ Use of results * t ■ 

I* 

F» New reports \ 
A.-F. ' ' j" t 

l'. 4 Written instructions * 

! - * ' # 1 
* 2. .Workshops. 

* * . * '* i ' 

\ a a . Answers to questions 

: I . ' • ..." 

b+A Training-films 

+ * ' ' ' % * '** \ * ' ' v' * ' * , * 
^; , ' c» (Jral presentations 
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Resolve longitudinal comparisons. *Our schools had a large/investment 
•in past years 1 test scores; therefore, elaborate measures were re- 
quired to preserve that longitudinal base. We opted for a , locally 
conducted equating study, then the schools were reorganized, by the ' • 
court for desegregation purposes, and most comparisons became mcfot. 

.A. Equating study 

1*. By the publisher / 

2. By the school system » 
a. Sample 

l . b.' Test * . 

* * • 1 s 

c. Analyze _ * 

B. Match old and new test' areas *^ 

; *' > 
Design the processing and scoring system. If you process and scor^ your 
own tests, this is another great opportunity to make^major improvements. 
We started out by listing every problem, error, suggestion, cjreara, etc. 
that we had ever encountered, and then set out. to afldress each one. We 
cut our turnaround tim§"(by 85%), our error rate, and our blood pressure 
by redesigning our entire processing and scoring system. . 

A. Scheduling - •* 

c 1. Document cleaning ' ~ . 

■* * * k ** 

1 2. Optical scanning time . » <? . 

-> • • 

J3. Keypunching time 

4. Mailing of jreports" » > s 9 

-5. Computer' time ^ * 

B. t Data" entry * ' r 
* ' ^l.< Keypunching < 4 ; 

• : • 

2. Optical spanning ■ 
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C. -Computer programming 

1* Notmp tables 1 * v K 

2. Correct' choice keys 
. 3. Ski'lis tables 
4. Scoring programs 
* 5* Report generation 



1.-5. v \ 

a. Ent&r 

b. Pjrptif* proof; *proof, proof... 

Dispose of old materials. Old materials are everywhere. Disposing of 
them is like trying to remove warts, ■ Usually out-of-date tests cannot 
be sold, or even given away. Recycling may be possible — if y<bu do not 
have to remcpve a billion staples. 



A. In jiffices 

B. On campuses' 

C. In storage 

► A. "**C • * * 



1", ' Sell 

2. / Give away* 



* 4 



3. Recycle - 

"4."" Save* for special 'purposes" ' J 

a. Special education practice v s I 

t * * 

0 b. Project evaluation . 

c« -Individual assessment * 

-\ d* -File copies ' ' * 

. " * . \ - \ - % 
Prepare- security measure's. An achievement test is a large, investment 5 
which can be, lost* if ^jsfecurity is loose-, We were surprised to find* 
copies in libraries/- schools, and offipes. We have &ad reasonable 
success- in limiting access to copies^in libraries and ^hoft circuiting, 
ord«rs to ; the publisher from school personnel. . 



\ % 'Ar M?±nif:ain\an inventory 



\ 



k. Restek^t availability 
; 1. University libraries 

; 2.; Schools 

\ 

i \ ■ 
■ 3* Centfal offices 



1 4. Review copies- - * 

\ * • * 

5. Educational resource Centers ' 

6. Publisher 

a. SaB\ple .copies 1 v 

* 4 b. . Purchase orders ' | • 

Obviously, the goal of this paper is to identify issues not to present solutions 
to ,them. After all, each schoo,l system's solutions musjt be unique to match 
local constraints. The next time that the Austin school system switches achieve- 
ment tests, we will be pulling this paper out of the file well in advance of the 
change over. \ 



